An optimized algorithm for Data Oriented Parsing

This paper presents an optimization of a syntactic disambiguation algorithm for Data Oriented Parsing (DOP) (Bod 93) in particular, and for Stochastic Tree-Substitution Grammars (STSGs) in general. The main advantage of this algorithm on existing alternatives ((Bod 93), (Schabes & Waters 93), (Sima'an et al. 94)) is that its time-complexity is linear, instead of square, in grammar-size (and cubic in sentence length). It is particularly suitable for natural language STSGs which have many deep elementary-trees and a small underlying Context-Free Grammar (CFG). A rst implementation of this algorithm is operational and is exhibiting substantial speed up in comparison to the unop-timized version. In addition to presenting the optimized algorithm, the paper reports experiments for measuring the disambiguation-accuracy, the expected sizes and the execution-times of various DOP models, which are projected from the ATIS domain. 1 Motivation Many models of natural language performance tend to train presupposed grammars in order to extend them probabilistically (e.g. (Schabes & Waters 93), (Black et al. 93)). In contrast , Data Oriented Parsing (DOP), suggested by Scha (Scha 90) and developed by Bod (Bod 92), projects an STSG directly from a given tree-bank. DOP projects an STSG by decomposing each tree in the tree-bank in all ways, at zero or more internal nodes each time, obtaining a set of constituent structures, which then serves as the elementary-trees set of an STSG. An STSG is basically a Context-Free Grammar (CFG) with \rules" (or \productions") which have internal structure i.e are (elementary-)trees. Deriving a parse for a given sentence in STSG is combining elementary-trees using the same substitution operation as used by CFGs. In contrast to CFGs, however, STSGs allow various derivations to generate the same parse. Crucial for natural language disambiguation, the set of trees generated by combining the elementary-trees of an STSG are not always gen-erateable by a CFG; thus, STSGs impose extra constraints on the generated structures. For selecting a distinguished structure from the space of generated structures for a given sentence, DOP assigns probabilities to the application of elementary-trees in derivations. The probability, which DOP inferres for each elementary-tree, is the ratio between the number of its appearances in the tree-bank (i.e. either as a tree or as a sub-tree) and the total number of appearances of all elementary-trees which share with it the same root non-terminal (see gure 1). A derivation's probability is then deened as the multiplication of the probabilities of …